1. Field of the Invention
The present invention relates to measuring the performance of wireless communications systems. More specifically, the present invention relates to systems and methods for calculating real-time performance and load statistics on such communications systems.
2. Description of the Related Art
In wireless telecommunication devices, such as cellular phones, PDAs, mini-laptops, and advanced pagers, the devices typically communicate over long distances by bridging telephone calls through existing cellular telephone networks and passing data packets across the network. These wireless devices often have significant data processing and computing capability and can accordingly send and receive software programs, in addition to voice, across the telephone network.
There exists a wireless telecommunication service that provides a quick one-to-one or one-to-many communication that is generically referred to as “Push to talk over cellular” (“PTT PoC,” “push to talk,” “PTT”) capability. The specific PTT group of recipient devices for the communicating wireless device is commonly set up by the carrier. A PTT communication connection is typically initiated by a single button-push on the wireless device that activates a half-duplex link between the speaker and each member device of the group and once the button is released, the device can receive incoming PTT transmissions. In some arrangements, the PTT speaker will have the “floor” where no other group member can speak while the speaker is holding the button. Once the speaker releases the PTT button, any other individual member of the group can engage their PTT button and take the floor.
It is useful to measure the performance metrics of a wireless communication system that processes calls between a plurality of wireless communications devices, such as a PTT system. Such a system is generally expected to provide service at a certain quality under a given load. This information can be used to provide information about the system status (load and performance of the system) to the network management interface, which can then be used to verify the expected system behavior. Most of the time, certain performance criteria need to be evaluated to verify the contractual agreement between a telecom operator and vendor. It can also be used to provide feedback to the system's call processing algorithm to ensure the necessary system stability and performance.
For instance, a system requirement may entail that for the 95th percentile, the system (such as a QUALCOMM QCHA™ system) will contribute no more than 150 ms to initial PTT latency for intra-regional QCHAT direct calls. Without providing a distribution of the measurement from the system, it is not possible to comprehensively verify this metric from the operational commercial network. It would therefore be useful to allow for such a metric to be calculated. Furthermore, it would be useful to provide this information on demand by using the standard network management interface example Simple Network Management Protocol (SNMP) command.
A significant concern that colors the ability of telecommunications systems to provide real-time performance data is the limitations of the systems themselves. Such a system is frequently under an extremely heavy load and has limited resources, both in computational power and memory. To this end, techniques of monitoring performance data must be efficient in at least these areas. Because a call processing server needs to be very fast and connect calls with very low latency, most techniques for measuring system performance are disfavored because they slow down the call processing server.
The prior art teaches taking performance data, storing it off the server, and later calculating the results. The prior art teaches away from real-time performance measurements because it teaches that this offline post facto technique is suitable for the application, when the technique is very slow. The prior art also teaches that only a very limited amount of data may be stored, such as one second's worth of data, and that the storage cannot scale up past a few minutes' worth of data. This limits the validity of measurements. The prior art also teaches using a rolling sum, which allows for producing an average quickly, but cannot be used to produce a maximum, minimum or 95th percentile. There exists a problem because existing systems do not have an efficient mechanism of providing a statistical distribution of the communication system's load and performance measurement in near-real-time.
An example in the prior art is a technique that uses a circular queue of a very small number of the most recent data points received, such as 100. For instance, each of 100 data points is represented by a 32-bit integer. However, if each of those integers was instead used to represent a range of values, and the value of that integer represented the number of data points that fall into that range, each of 100 32-bit integers may represent a bin, with each bin allowing for over 4 billion (2^32) measurements. A small sacrifice in granularity allows for a significant increase in the number of data points that can be stored.
Present telecommunications systems are able to provide an average, minimum and maximum value of system load and system performance metrics based on a limited number of samples or data collected over a limited period of time. This usually takes the form of a running average, where the system stores a sum total and number of calls, as well as a minimum and maximum value observed. It is a limitation of this technique that the granularity of the original data is lost—that is, while a running average value may be calculated, the individual measurements are lost so that, for instance, a 95th percentile of those measurements cannot be calculated. Furthermore, because the granularity is lost, it is not possible to obtain the actual distribution of the load and performance metrics of the system. Additionally, frequently an average, minimum or maximum may not meet the desired need to verify system performance (such as when the system performance metric is based on that 95th percentile).
Without a finer-grain measurement of the performance, performance metrics may appear to fall within an acceptable range when a sizeable portion of the user base may experience unacceptable performance (a few extremely quickly connected calls brings down the average call connection time while the majority of users are experiencing a slow call connection time). As such, it is necessary to know the distribution of system performance metrics in order to evaluate the true performance of the system.
There is not a system currently that allows one to measure the percentile, such as the 95th percentile, of measured data based on either a number of most recent calls, such as 200, or a most recent time slice, such as 30 minutes. 30 minutes of calls during a high use hour on a big system may represent several million calls. Furthermore, to store all those calls takes memory resources away from the call processing part of the system, which reduces system performance. To that end, techniques and systems that would allow for such measurements would be an improvement over the prior art.